BLD/DEV: use meson-python, add a basic Pixi workspace by lucascolley · Pull Request #373 · data-apis/array-api-compat

lucascolley · 2026-01-06T22:30:28Z

Clearly plenty more work to do to get this up to par with array-api-extra, but I think that merging something like this would be a good start.

Follow-ups:

Convert CI to Pixi
add linting

ev-br

Let's maybe revive this if you're interested @lucascolley ? Sorry it took me long to get back to this.

I trust you on anything pixi. If dynamic versioning gets in the way, I'd be happy to see the back of it, static is indeed much nicer.

What needs to be done if we change the versioning:

purge any mentions of setuptools_scm and tool.setuptools.dynamic from pyproject.toml
update the releasing docs to mention that the version needs to be bumped in both __init__.py and pyproject.toml it currently says "update the version in __init__.py"
cut a dummy release locally (python -m build . && twine check --strict dist/
make a dummy TestPyPI release to verify it all works

lucascolley · 2026-03-16T21:12:55Z

❯ pixi exec -w python-build python -m build .
...
❯ pixi exec twine check --strict dist/*.tar.gz dist/*.whl
Checking dist/array_api_compat-1.13.0-py3-none-any.whl: PASSED
Checking dist/array_api_compat-1.13.0.tar.gz: PASSED

make a dummy TestPyPI release to verify it all works

do you want me to do this?

lucascolley · 2026-03-16T21:38:28Z

I implemented the switch from setuptools to meson-python

ev-br

Agreed that the change to meson-python is nice.

I actually wonder about vendoring scenarios. When e.g. SciPy vendors array-api-compat, it has to list the sources in the meson.build file. Now that array-api-compat has its meson.build file with the listing, can scipy reuse it? This is not blocking this PR of course, am just thinking out loud.

make a dummy TestPyPI release to verify it all works

Thinking about it, to test trusted publishing it needs to be done from main/tag etc. Presumably trusted publishing is decoupled enough, so that it's more work than justified now. So checking that a release wheel looks reasonable locally is probably sufficient.

lucascolley · 2026-03-17T09:20:06Z

Yes, this is the exact context within which I started this work: https://github.com/scipy/scipy/blob/main/meson.build#L216-L246

ev-br · 2026-03-17T11:38:12Z

Tried test-driving it with my usual workflow:

$ pip install -e . --no-build-isolation
$ pushd ../array-api-tests
$ python
>>> from array_api_compat import torch as xp
...
ModuleNotFoundError: No module named 'array_api_compat.common'

In fact, non-editable installs fail in the same way as well. What gives?

lucascolley · 2026-03-17T12:05:35Z

Hmm the problem is reproducible even in the default Pixi environment via pixi run python, I'm baffled at the minute.

lucascolley · 2026-03-17T12:43:46Z

+install_symlink(
+  '_compat',
+  install_dir: 'array_api_compat/vendor_test/vendored',
+  pointing_to: '../../',
+)


The problem was that I had a dodgy workaround for the symlink. Hopefully now this works.

@rgommers can you see what I'm doing wrong with the symlink here? Logs like https://github.com/data-apis/array-api-compat/actions/runs/23195083314/job/67401163424?pr=373 seem to show that it is broken.

I've tried poking around with meson introspect but I can't figure it out.

Symlinks aren't supported in wheels, so this looks unfamiliar and is probably not what you want.

It is possible to have a symlink in the source tree, but it gets replaced by a copy in an sdist/wheel.

9120e06 (this PR) gets rid of the symlink. However, this does not seem to work for editable builds. Is there something in Meson's Python module that can mimic configure_file(copy: true) like this?

For context, it seems like pointing to the same file twice with py.install_sources means that it is only installed once.

despite my sunk-cost into making this work for wheels only, we should probably just give up on the symlink and use a Pixi task to populate the directory at test-time, like https://github.com/data-apis/array-api-extra/blob/1c5bb9d65f75132196650de2ca966e2fc36ac15c/pyproject.toml#L146. I think we are pushing meson-python slightly past its limits by trying to mimic the symlink.

I had hoped that we could merge this without me having to convert CI over to using Pixi yet, but maybe not...

I'm a little confused here about what triggered all this, because there's a lot going on in this PR:

The move to src/ layout, if desired, might be best to land standalone, together with a commit to not affect git blame

The vendor test is new, and is what seems to be the motivation for the symlink-like thing. Can't this be done inside the test itself, rather than in the source tree?

Switching over to meson-python, nice (for SciPy/scikit-learn and the vendoring use case)

Adding a Pixi config

The vendor test is new, and is what seems to be the motivation for the symlink-like thing. Can't this be done inside the test itself, rather than in the source tree?

the vendor test is not new, you can see the symlink at https://github.com/data-apis/array-api-compat/tree/main/vendor_test/vendored. The challenge I was trying to get through was dealing with the symlink in meson-python. It would be nicer to just get rid of the symlink and copy things over at test-time, like https://github.com/data-apis/array-api-extra/blob/231b8a50d9baa22ccebdfee571472f0ce46a8a29/pyproject.toml#L145-L146.

Ah never noticed (or, no longer remembering) - it's missing from the sdist, which is unsurprising but not ideal.

It would be nicer to just get rid of the symlink and copy things over at test-time

agreed

lucascolley · 2026-05-25T10:25:25Z

@ev-br please could you let me know if you have any remaining concerns? thanks!

ev-br

A couple of questions below.
Other than that:

pixi changes I happily defer to you Lucas :-).
dynamic versioning -> static versioning change is great, let's rid of it.
meson is a bit of an overkill for a pure python package --- I'd use flit for a pure-python package with static versioning, but OK if we want to use meson throughout;
src layout: is it needed (why?) or is it just because why not (also why though?)
could you double-check what scipy and other vendorers need to do to adapt?

ev-br · 2026-05-25T11:44:05Z

 def test_vendoring_dask():
    pytest.importorskip("dask")
-    from vendor_test import uses_dask
+    from . import uses_dask


Could you explain this and related changes please?

test_vendoring was moved from the tests directory to the vendor_tests directory, and this line is updated with the appropriate relative import. This matches what works well for us at https://github.com/data-apis/array-api-extra/tree/main/vendor_tests.

ev-br · 2026-05-25T11:51:46Z

@@ -1,3 +1,6 @@
+pixi.lock
+vendor_tests/array_api_compat


Could you explain these two?

ignoring pixi.lock: naively, I'd expect that if some dependency breaks, we'd want to bisect pixi.lock updates. From experience, is this not a useful workflow?

Never saw any issues with vendor_tests/array_api_compat (or the very directory for that matter), why do we want to ignore it now?

I'm happy to commit pixi.lock. Without committing it, each individual contributor may have different environments on their machine, although they will all still be compatible with pixi.toml.

I was making the minimal change in case there were objections to committing the lock file, but if there aren't, then great, let's commit it.

Never saw any issues with vendor_tests/array_api_compat (or the very directory for that matter), why do we want to ignore it now?

The vendoring tests are now changed from the previous symlink at vendor_test/vendored/_compat to a cp command that populates this newly gitignored directory. We discussed above that we wanted to get rid of the symlink.

There is a Pixi task for this cp command that runs automatically with pixi run tests-vendor, and it is currently hardcoded in CI. I've marked converting CI to Pixi for a follow-up.

lucascolley · 2026-05-25T12:23:05Z

meson is a bit of an overkill for a pure python package --- I'd use flit for a pure-python package with static versioning, but OK if we want to use meson throughout;

I agree that in general meson-python is overkill for a pure Python package, but in this specific case where we want to make it easily vendor-able by projects using meson-python, it makes sense

src layout: is it needed (why?) or is it just because why not (also why though?)

I think I originally introduced it because it kept hatchling on default settings happy. But indeed there are reasons why it is valuable, e.g. avoiding accidental use of relative imports. Let me know if you have more questions after reading https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/.

could you double-check what scipy and other vendorers need to do to adapt?

The work is already done for SciPy, this just enables us to delete https://github.com/scipy/scipy/blob/main/subprojects/array_api_compat/meson.build. Other vendorers can copy SciPy's approach.

lucascolley commented Jan 6, 2026

View reviewed changes

Comment thread pyproject.toml Outdated

lucascolley commented Jan 6, 2026

View reviewed changes

Comment thread pixi.lock Outdated

ev-br requested changes Mar 16, 2026

View reviewed changes

Comment thread pyproject.toml Outdated

lucascolley self-assigned this Mar 16, 2026

lucascolley force-pushed the pixi branch 2 times, most recently from 6c2a523 to 82e4abe Compare March 16, 2026 21:06

lucascolley force-pushed the pixi branch from 82e4abe to 440e728 Compare March 16, 2026 21:14

lucascolley requested a review from ev-br March 16, 2026 21:43

lucascolley changed the title ~~DEV: add a basic Pixi workspace~~ BLD/DEV: use meson-python, add a basic Pixi workspace Mar 16, 2026

ev-br requested changes Mar 17, 2026

View reviewed changes

Comment thread docs/dev/releasing.md

lucascolley commented Mar 17, 2026

View reviewed changes

Comment thread docs/dev/releasing.md

lucascolley requested a review from ev-br March 17, 2026 10:03

lucascolley removed their assignment Mar 17, 2026

lucascolley force-pushed the pixi branch 3 times, most recently from 93ba85a to 2470147 Compare March 17, 2026 12:42

lucascolley commented Mar 17, 2026

View reviewed changes

lucascolley force-pushed the pixi branch 5 times, most recently from c47c034 to 8f4f87e Compare May 25, 2026 10:16

BLD/DEV: use meson-python, add a basic Pixi workspace

a5b0c04

lucascolley force-pushed the pixi branch from 8f4f87e to a5b0c04 Compare May 25, 2026 10:25

ev-br reviewed May 25, 2026

View reviewed changes

update release docs prose

de51742

lucascolley requested a review from ev-br May 25, 2026 15:03

Conversation

lucascolley commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ev-br left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lucascolley commented Mar 16, 2026

Uh oh!

lucascolley commented Mar 16, 2026

Uh oh!

ev-br left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lucascolley commented Mar 17, 2026

Uh oh!

Uh oh!

ev-br commented Mar 17, 2026

Uh oh!

lucascolley commented Mar 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lucascolley commented May 25, 2026

Uh oh!

ev-br left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lucascolley commented May 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lucascolley commented Jan 6, 2026 •

edited

Loading